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Application of Mathrubootham Janakiraman et al. Art Unit 2155 

Serial No. 09/931,211 
Filed August 16, 2001 
Confirmation No. 2348 

For METHOD AND SYSTEM FOR SELECTIVELY VIEWING PARTICIPANTS OF A 

MULTIMEDIA NETWORK CONFERENCE 
Examiner Asad M. Nawaz 

APPEAL BRIEF 

This is an appeal from the final rejection of the claims of the above-referenced 
application made in the Final Office action dated May 30, 3008. A Notice of Appeal and a Pre- 
Appeal Brief was filed on November 25, 2008. A Notice of Panel Decision from Pre- Appeal 
Brief Review as mailed on January 14, 2009, concurring in the rejection of claims 10-16, 18 and 
24-35. 

The appeal brief fee in the amount of $540.00 is submitted herewith. 

I. REAL PARTY IN INTEREST 

The real party in interest in connection with the present appeal is Microsoft Corporation 
of One Microsoft Way, Redmond, Washington, 98052, a corporation of the state of Washington, 
owner of 100 percent interest in the pending application. 

II. RELATED APPEALS AND INTERFERENCES 

Appellant is unaware of any pending appeals, interferences, or judicial proceedings that 
may be related to, directly affect or be directly affected by, or have a bearing on, the Board's 
decision in the pending appeal. 

III. STATUS OF CLAIMS 

Claims 10-16, 18, and 24-35, as set forth in the Claims Appendix, are currently pending 
in the application for consideration. Claims 1-9, 17, and 19-23 have been canceled. 
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Claims 10-16, 18, and 24-35 stand rejected. The rejection of each of these claims is 
being appealed. 



IV. STATUS OF AMENDMENTS 

Amendment I was filed on September 10, 2008 responding to the final Office action 
dated May 30, 2008. Amendment I was not entered because the Examiner deemed them to 
change the scope of the limitations. Therefore, the current status of the claims reflects the listing 
of the claims as amended in Amendment H (filed February 20, 2008). 

V. SUMMARY OF CLAIMED SUBJECT MATTER 

The following summary correlates claim elements to embodiments described in the 
application specification, but does not in any manner limit claim interpretation. Rather, the 
following summary is provided only to facilitate the Board's understanding of the subject matter 
of this appeal. The elements of the claims presented in this section have been bolded and 
italicized for convenient identification. 

Aspects of the present invention provides "a method and system for intelligently selecting 
a single video stream from video streams originating from multiple participants of a multimedia 
network conference and sending the selected video stream to a client for viewing." 1 "In 
accordance with the invention, periodically, each of the participants is assigned a weight that is 
dynamically determined" 2 as "a function of the participants' activity state variables . . . and a set 
of tunable parameters called 'participant selection control parameters.'" 3 "The participant with 
the highest weight among all the participants is then selected for viewing by the client, i.e., the 
video stream from that participant is sent to the client." 4 

Claim 10 is directed to a system for conducting a multimedia conference. 5 The system 
includes a plurality of participants each providing multimedia conferencing data including a 
video signal and an audio signal. For example, the embodiment of the invention illustrated in 

1 Application, page 3, lines 11-15. 
Application, page 12, line 27 to page 13, line 3. 

3 Application, page 17, lines 3-6. 

4 Application, page 3, lines 22-25. 

See Application, page 3, lines 11-14 ("the present invention provides a . . . system for intelligently selecting a 
single video stream from video streams originating from multiple participants of a multimedia network 
conference.") 
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Figure 2 of the present application "shows two multicast-capable clients 102, 104 connected to 
the multicast network 100." 6 Each of the two multicast clients provides outgoing "audio and 
video streams 1 10 ... for delivery to all the other participants." 7 The system of claim 10 also 
includes a client in conference with the participants, and is capable of receiving the video 
signal corresponding to one of the participants at a time. For example, the embodiment of the 
invention illustrated in Figure 2 includes a client (e.g., 1 12, 1 14) who "can only receive the video 
stream of one participant at a time." 8 

The system of claim 10 includes a bridge server connected to the participants through a 
network and having a point-to-point connection with the client. As described by the 
specification, embodiments of the bridge server allow "a client (e.g., client 112) that is not 
multicast-capable or multicast-connected [to] participate in a network conference." 9 As 
explained by the present application in reference to FIG. 2, "the bridge server 120 is connected to 
the multi-cast network 100" 10 and "functions as a proxy for connecting the client 1 12 to a 
multimedia conference." 11 Specifically, "[w]hen the client 112 wants to participate in a 
multimedia network conference, it places a point-to-point call to the bridge server 120." 12 
Accordingly, "[t]he bridge server, on behalf of the client, then joins the multicast group defining 
the conference." 13 The bridge server sends a video stream to the client which "contains only one 
video substream representing only one of the participants." 14 

In order to select the video stream for sending to the client from the plurality of video 
streams originating from the participants of the network conference, the system includes a 
participant state table and a participant selection control parameter stored in memory. The 
participant state table indicates] an activity state variable for each participant. 15 The activity 
state variable include fesj values and statistics associated with the participant's video signal 
and audio signal, 16 "such as whether the participant is being shown to the client, how long the 



6 Application, page 10, lines 9-11. 

7 Application, page 10, lines 19-21. 

8 Application, page 12, lines 26-27; See also, page 9, lines 25-27. 

9 Application, page 1 1, lines 9-11. 

10 Application, page 11, lines 11-12. 

11 Application, page 11, lines 14-15. 
Application, page 11, lines 15-18. 
Application, page 11, lines 18-20. 

14 Application, page 12, lines 16-19. 

See Application, page 13, lines 21-23; page 14, lines 24-27; Figure 4. 
16 Application, page 14, lines 25-27. 
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participant has been shown or not shown, etc." 17 "The activity states of [each] participant are 
then used in the weight assignment process discussed below. 

The participant selection control parameter is received when the multimedia conference 
is set up l% for tuning a video switching stream behavior 19 during the conference. As shown in 
Table 1 of the present application, each received participant selection control parameter has a 
constant/fixed value which affects the outcome of the weight computation discussed below 20 . 
Accordingly, each participant selection control parameter has a static display constraint on a 
selection of a video signal? 1 

For example, the exemplary participant selection control parameters in Table 1 include a 
"Minimum Shown Time" and a "Minimum Shown Time If Active." The "Minimum Shown 
Time" control parameter specifies the minimum period of time (8 seconds) that a selected 
participant's video stream will be displayed on the client stream. 22 The "Minimum Shown Time 
If Active" control parameter specifies the minimum period of time (15 seconds) that a selected 
participant's video stream will be displayed on the client screen if the participant is still talking 
for this period of time." 23 These two parameters "help to prevent a flurry of abrupt jumps from 
one participant to another. For example, if these parameters are not used and the switching is 
based only on which participant happens to be making the loudest sound, then the screen image 
may be switched back and forth too quickly and too frequently between the talking participants, 
resulting in an unpleasant client experience." 24 

During the conference, the bridge server receivfesj simultaneously the multimedia 
conferencing data including the video signal from each of the participants 25 and updatefsj the 
activity state variable stored in the memory for each participant in the participant state table 
according to changes in a data information and a control information of the participant's 
video signal and audio signal. 26 The bridge server periodically computfesj a weight of[] each 



17 Application, page 3, lines 18-20. 

18 Application, page 19, lines 8-9. 

19 See Application, page 17, lines 6-9; page 19, lines 5-6. 

20 Application, page 17, lines 3-6. 
Application, page 17, lines 3-6. 
Application, page 18, Table 1, Row 2. 
Application, page 18, Table 1, Row 3. 

24 Application, page 21, lines 12-19. 

Application, page 11, lines 21-26; See also Figure 3. 
26 Application, page 13, lines 11-23 ("In this embodiment, several participant events are defined and used to update 
activity states of the participant. The activity states of the participant are then used in the weight assignment 
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participant based on the activity state variable of[] each participant and the participant 
selection control parameter} 1 

In particular, the bridge server assignfsj a predetermined weight to at least one of the 
plurality of participants for a duration specified by the static display constraint. For example, 
for the participant is being shown (step 184), it is determined whether the 

SecsSinceLastStartedShowing value is less than the Minimum Shown Time (step 192). If so, the 
weight is set to be MAXWEIGHT (step 194), which is a very large value, to ensure that this 
participant will be selected. This guarantees that a participant, once selected for viewing, will be 
shown for at least the Minimum Shown Time (e.g., 8 seconds)." 28 

After the bridge server has computed a weight for the participants, the bridge server 
identify fiesj a participant having a highest weight among the participants, and selectfsj . . . the 
video signal corresponding to the identified participant having the highest weight for 
transmission to the client for viewing} 9 

Claim 24 is directed to a method for selecting one video signal from a plurality of video 
signal for forwarding to a client} 0 Each video signal correspond[s] to a participant of multiple 
participants of a multimedia conference. 31 The method includes receiving a participant 
selection control parameter for the multimedia conference when the conference is being set 
up} 2 The participant selection control parameter hafsj a static display constraint of selecting 
the one video signal. 33 For example, as discussed above, the exemplary participant selection 
control parameters in Table 1, "Minimum Shown Time" and "Minimum Shown Time If Active" 
specify minimum times for showing a participant's video stream. 34 These two parameters "help 
to prevent a flurry of abrupt jumps from one participant to another. For example, if these 
parameters are not used and the switching is based only on which participant happens to be 



process. In this regard, the multimedia streams received by the bridge server from the multicast group include both 
data and control information. In response to changes in both of these pieces of information, the multicast conference 
server 122 generates the participant events. As a part of handling these events, the multicast conference server 122 
updates a participant state table 150 associated with the conference."). 

27 Application, page 17, lines 3-6; page 13, lines 1-2. 

28 Application, page 20, lines 19-26; See also Figure 6. 

29 Application, page 3, lines 22-25; page 16, lines 16-19; page 16, line 24 to page 17, line 1. 

30 Application, page 12, lines 10-12. 

31 Application, page 12, lines 4-6. 

32 Application, page 19, lines 8-9. 

33 Application, page 17, lines 3-6. 

34 Application, page 18, Table 1, Rows 2-3. 
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making the loudest sound, then the screen image may be switched back and forth too quickly and 
too frequently between the talking participants, resulting in an unpleasant client experience." 35 

The method of claim 24 includes receiving simultaneously multimedia conferencing 
data . . . that includ[es] the plurality of video signals from the participants 26 and monitoring 
participant events of the multimedia conference 2 '. The participant events [are] generated in 
response to changes in the data information and the control information of the multimedia 
conferencing data. 3S For example, the participant events may include "NewSub Stream" for 
indicating that the participant stated sending video and "SubStreamRemoved" for indicating that 
the participant stopped sending video. 39 

Additionally, the method of claim 24 includes providing a participant state table . . . 
indicating an activity state variable for each participant of the multimedia conference 40 and 
updating at least one of the activity state variables in the participant state table according to 
the participant events. The activity state variable includ[es] values and statistics associated 
with the participant's multimedia conference data, 41 "such as whether the participant is being 
shown to the client, how long the participant has been shown or not shown, etc." 42 

The method of claim 24 includes periodically computing a weight for each of the 
participants based on the activity state variable of[] each participant and the participant 
selection control parameter 43 and identifying a participant having a highest weight among the 
participants. 44 Accordingly, the [] video signal corresponding to the participant having the 
highest weight is selectfedj from the received multimedia conference data . . . for viewing by 
the client 45 



35 Application, page 21, lines 12-19. 

36 Application, page 11, lines 21-26; See also Figure 3. 

37 Application, page 3, lines 16-20. 

38 Application, page 13, lines 11-23 ("In this embodiment, several participant events are defined and used to update 
activity states of the participant. The activity states of the participant are then used in the weight assignment 
process. In this regard, the multimedia streams received by the bridge server from the multicast group include both 
data and control information. In response to changes in both of these pieces of information, the multicast conference 
server 122 generates the participant events. As a part of handling these events, the multicast conference server 122 
updates a participant state table 150 associated with the conference."). 

39 Application, page 13, lines 26-27. 

40 See Application, page 13, lines 21-23; page 14, lines 24-27; Figure 4. 

41 Application, page 14, lines 25-27. 

42 Application, page 3, lines 18-20. 

43 Application, page 17, lines 3-6; page 13, lines 1-2. 

44 Application, page 3, lines 22-25; page 16, lines 16-19. 

45 Application, page 3, lines 22-25; page 16, line 24 to page 17, line 1. 
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VI. 



GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 



A. Appellants appeal the rejections of claims 10-16, 18, and 24-35 under 35 U.S.C. 
§1 12, second paragraph, as being indefinite for failing to particularly point out and distinctly 
claim the subject matter which applicant regards as the invention. 

B. Appellants appeal the rejections of claims 10-16, 18, and 24-35 under 35 U.S.C. 
§ 103(a) as being obvious over Sandvoss et al U.S. Pat. No. 5,745,380 (Sandvoss) in view of 
OkamuraU.S. Pat. No. 6,178,424. 

VII. ARGUMENT 

A. Claims 10-16, 18, and 24-35 are patentable under 35 U.S.C. $112 as being definite and 
particularly pointing out and distinctly claiming the subject matter of the claimed 
invention. 

No indefmiteness or ambiguity is created by "the outcome of a weight computation" and 
"the data information and the control information of the participant's video signal and audio 
signal, specified by claim 10. The outcome is an inherent aspect of a weight computation and 
the data information and the control information are inherent aspects of the video and audio. 
Thus, these recitals do not require antecedent basis. Similarly, the terminology of claim 24 is 
clear and unambiguous for the same reasons. Thus, the rejection under 35 U.S.C. § 1 12, second 
paragraph should be reversed. 

It is noted that Appellants attempted to resolve the 112 rejection only in order to reduce 
the issues on appeal by filing Amendment I (filed September 10, 2008). For example, the 
Appellants submitted changing "the outcome" to "an outcome" in claim 10. In addition, the 
Appellants submitted similar minor claim language changes to claims 10 and 24 to simply avoid 
the 112 issues (which were raised by the Examiner for the first time in the final Office action). 
Unfortunately, the Examiner refused to enter Amendment I (filed September 10, 2008) 
disingenuously arguing that the minor amendments raised new issues. 

In the event that the Board believes that the claim language needs clarification, 
Appellants request that the Board remand the application to the Examiner so that Appellants can 
file a "housekeeping" amendment to resolve any minor 112 language issues. 
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B. Claims 10-16, 18, and 24-35 are patentable under 35 U.S.C. §103(a) as being non- 



obvious over the Sandvoss in view of Okamura. 



B(l). Claim 10 

(i). Sandvoss and Okamura, alone or in combination, fail to teach each and every 
element of claim 10 

Claim 10 relates to a "system for intelligently selecting a single video stream from video 
streams originating from multiple participants of a multimedia network conference and sending 
the selected video stream to a client for viewing." 46 As recited by claim 10, the system 
comprises, among other things, the combination of (a) a "participant selection control parameter 
having a static display constraint on a selection of a video signal," (b) "a participant state table . . 
. indicating an activity state variable for each participant" wherein the "activity state variable 
include[s] values and statistics associated with the participant's video signal and audio signal; 
and (c) "a bridge server connected to the participants through a network" and "assigning a 
predetermined weight to at least one of the plurality of participants." 

As recited by claim 10, the bridge server "update[s] the activity state variable ... for each 
participant in the participant state table" and "periodically comput[es] a weight of said each 
participant based on the activity state variable of said each participant and the participant 
selection control parameter." In particular, the bridge server "assign[s] a predetermined weight 
to at least one of the plurality of participants for a duration specified by the static display 
constraint." Accordingly, the bridge server "identify[ies] a participant having a highest weight 
among the participants, and select[s] ... the video signal corresponding to the identified 
participant having the highest weight for transmission to the client for viewing." 

For claimed subject matter to be prima facie obvious under 35 U.S.C. § 103(a) in view of 
prior art, the prior art references must individually or in combination disclose or suggest each of 
the requirements of the claim. 47 As explained below, neither Sandvoss nor Okamura, alone or in 



Application, page 3, lines 11-15. 
MPEP §§ 2143-2143.01. 
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combination disclose or suggest each of the requirements of claim 10. Thus, the rejection of 
claim 10 under 35 U.S. C. § 103(a) must be withdrawn. 

(i)(a). Sandvoss and Okamura, alone or in combination, fail to teach the participant 
selection control parameter having a static display constraint of claim 10 

As repeatedly pointed out by Appellants in Amendments G, H, and 1, 48 Sandvoss and 
Okamura fail to teach or disclose a "participant selection control parameter having a static 
display constraint on a selection of a video signal" as recited by claim 10. The participant 
selection control parameter recited by claim 10 is "received when the multimedia conference is 
being set up" and remains at the fixed value unless changed by the user. For example, the 
exemplary participant selection control parameters in Table 1 of the present application include a 
"Minimum Shown Time" and a "Minimum Shown Time If Active." The "Minimum Shown 
Time" control parameter specifies the minimum static period of time (8 seconds) that a selected 
participant's video stream will be displayed on the client stream. 49 The "Minimum Shown Time 
If Active" control parameter specifies the minimum static period of time (15 seconds) that a 
selected participant's video stream will be displayed on the client screen if the participant is still 
talking for this period of time." 50 Thus, the participant selection control parameter recited by 
claim 10 has a static value, as opposed to a dynamic value, and thus does not change unless 
specifically re-configured by a user. 

The Office incorrectly asserts that Sandvoss discloses the static display constraint 
requirement of claim 10 by stating that Sandvoss teaches that the amplitude of an audio 
substream is used in a weighting function computed to select a multimedia signal. 51 In contrast 
to the Office assertions, Sandvoss discloses "the priorities of the streams are determined by a 
weighting function . . . comprising the amplitude loud(si,t) of an audio substream and a 
parameter A(si) which describes the last time when a stream has been active." 52 However, these 
variables are both dynamic, i.e., changing over time based on the input signals. Thus, Sandvoss 
fails to teach a "participant selection control parameter having a static display constraint on a 

48 Amendment G filed August 9, 2007; Amendment H filed February 20, 2008; Amendment I filed September 10, 
2008. 

49 Application, page 18, Table 1, Row 2. 

50 Application, page 18, Table 1, Row 3. 

51 Final Office action, page 3. 

52 Sandvoss, col. 5, lines 36-45. 
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selection of a video signal" as recited by claim 10. Moreover, even were the amplitude and the 
time that a last stream was active considered a static display constraint, these variables of 
Sandvoss are not received when the multimedia conference is being set up. Thus, Sandvoss 
cannot teach or suggest the recited features of claim 10. 

In fact, Sandvoss teaches away from using static display constraints received when the 
conference is set up to select a video stream by teaching that the priority of the streams are 
determined by a weighting function that is only dependent on dynamic values determined during 
the conference. In particular, Sandvoss states that "the weighing function is only dependent on 
the corresponding audio substreams and the time." 53 Since Sandvoss teaches selecting a 
multimedia stream based only on dynamic variables, Sandvoss fails to reduce the "flurry of 
abrupt jumps from one participant to another" 54 as provided by the present invention. The 
present application explains that if parameters with static display constraints are not used and the 
switching is based on which participant happens to be making the loudest sound as taught by 
Sandvoss, "then the screen image may be switched back and forth too quickly and too frequently 
between the talking participants, resulting in an unpleasant client experience." 55 

Okamura does not cure the insufficient and contrary teachings of Sandvoss with regard to 
the participant selection control parameter. As detailed below in Section B(2), Okamura teaches 
selecting users to receive information distributed from a distribution database based on user 
attributes and feedback provided by the user. 56 Okamura does not select information from one 
participant for transmission to a user. Moreover, nothing in Okamura teaches or suggests that 
the information includes a multimedia stream, such as a video stream or an audio stream. 
Accordingly, like Sandvoss, nothing in Okamura teaches or suggests a "participant selection 
control parameter having a static display constraint on a selection of a video signal" as recited by 
claim 10. 

(i)(b). Sandvoss and Okamura, alone or in combination, fail to teach the participant 
state table of claim 10 



Sandvoss, col. 5, lines 44-46. 
54 Application, page 21, lines 13-14. 

Application, page 21, lines 14-19. 
56 Okamura, col. 6, lines 11-25; FIG. 1; col, 7, lines 5-10; col. 12, lines 22-43. 
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The Office correctly acknowledges that Sandvoss fails to disclose each of the 
requirements of claim 10. Specifically, the final Office action admits that "Sandvoss does not 
explicitly indicate providing a participant state table indicating an activity state variable for each 
participant." 57 

Okamura fails to cure the admitted deficiencies of Sandvoss. In particular, Okamura fails 
to teach or suggest "a participant state table . . . indicating an activity state variable for each 
participant" as recited by claim 10. Figure 4 of the present application illustrates an exemplary 
participant state table which includes an "Activity State Information" field to keep track of the 
activity state variables for the participant. 58 As recited in claim 10, the activity state variables for 
each participant "include[] values and statistics associated with the participant's video signal and 
audio signal." For example, as described in the present application, the activity state variables 
may include "SecSeince Last StoppedShowing" for indicating the "time in seconds since [a] 
participant was last shown" 59 and/or "IsTalking" for "indicating whether [a] participant is 
currently talking" 60 . As additionally specified in claim 10, the activity state variables for a 
participant are used to compute a weight of the participant, 61 and, in turn, to determine whether 
the participant's video signal is selected for transmission to the client 62 . 

Although Okamura includes the term "activity state" as pointed out by the Office, 63 
Okamura teaches a completely different type of "activity state." Claim 10 explicitly recites that 
the activity state variable includes "values and statistics associated with the participant's video 
signal and audio signal." In contrast, Okamura merely teaches an information distributing 
system, connected to a variety of database systems, for distributing information, such as 
advertising information, fetched out of the database to selected recipient users. 64 Okamura 
includes the term "activity state" to refer to feedback from a user indicating whether information 



57 Final Office action, dated May 30, 2008, page 4. 

58 Application, page 14, lines 24-27. 

59 Application, page 15, lines 5-6. 

60 Application, page 15, lines 16-17. 

61 Claim 10 recites "periodically computing a weight of said each participant based on the activity state variable of 
said each participant." 

62 Claim 10 recites "identifying a participant having a highest weight among the participants, and selecting from the 
received multimedia conferencing data the video signal corresponding to the identified participant having the 
highest weight for transmission to the client for viewing." 

63 Final Office action, dated May 30, 2008, page 4. 

64 Okamura, col. 12, lines 22-43. 
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previously distributed to the user from the distribution system was "useful or useless." 65 Thus, 
the activity state of Okamura is user feedback regarding the usefulness of information previously 
received by the user, and not information transmitted by the user. Such user feedback is 
completely different from the activity state variable of claim 10 which is recited as describing the 
multimedia signal being transmitted by a participant. Specifically, claim 10 explicitly recites 
that the activity state variable includes values or statistics associated with the participant's video 
signal and audio signal. Additionally, the activity state (user feedback) of Okamura is used to 
determine whether or not to send the same user similar information items in the future, 66 rather 
than to determine whether to send information to another user. In particular, the activity state of 
claim 10 is used to determine whether the participant's video signal is transmitted to another 
participant (i.e., claim 10 recites that the bridge selects the participant's video signal having the 
highest weight for transmission to the client). 

Accordingly, Okamura clearly fails to disclose or teach the activity state variable 
explicitly recited by claim 10. Furthermore, even if Okamura was interpreted to teach such an 
activity state variable, Okamura fails to teach "a participant state table" indicating an activity 
state variable for each participant as required by claim 10. Okamura is entirely silent on a 
participant state table. After admitting that Sandvoss fails to disclose the participant state table, 
the Office has failed to point to even a suggestion thereof by Okamura. Thus, Okamura, like 
Sandvoss fails to disclose or suggest the "participant state table . . . indicating an activity state 
variable for each participant, said activity state variable including values and statistics associated 
with the participant's video signal and audio signal" as recited by claim 10. 

(i)(c). Sandvoss and Okamura, alone or in combination, fail to teach the bridge server 
of claim 10 

As noted above, the Office correctly acknowledges that Sandvoss fails to disclose each of 
the requirements of claim 10. Specifically, the final Office action admits that "Sandvoss does 



65 Okamura, col. 6, lines 20-24; col. 10, lines 23-26. 

66 Okamura, col. 6, lines 11-25; FIG. 1; col, 7, lines 5-10. 
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not explicitly indicate ... assigning a predetermined weight to at least a participant for duration 
previously set." 67 

Contrary to the Office's assertion, Okamura does not cure this failure of Sandvoss as 
Okumura does not teach or suggest a bridge server "assigning a predetermined weight to at least 
a participant for a duration previously set" as required by claim 10. The Office asserts that this 
feature is disclosed in Okamura at col. 2, lines 14-41. 68 However, this portion of Okamura 
merely explains that predetermined information attributes (e.g., keyword contained in the 
information, information provider, location of information) 69 are used to characterize information 
and the information is distributed to users based on user attributes (e.g., name, address, 
subscribed newspaper/magazine) 70 and the activity state (user feedback) of the user discussed 
above. Nothing in this cited portion, or any other portion of Okamura, even suggests that 
predetermined weights are assigned to a user, much less that the weights are assigned for a 
duration previously set. Thus, Okamura, like Sandvoss fails to teach or suggest "assigning a 
predetermined weight to at least a participant for a duration previously set" as required by the 
bridge server in claim 10. 

Thus, Sandvoss and Okamura, alone and in combination fail to teach or suggest each 
and every element of the invention recited by claim 10. Accordingly, Appellants respectfully 
submit that claim 10 is non-obvious and patentable over the cited art and that the rejection of 
claim 10 under 35 U.S. C. § 103(a) should be withdrawn. 

(ii). The Office fails to provide articulated reasoning to support obviousness 

Additionally, "[t]he test for obviousness is what the combined teachings of the references 
would have suggested to one of ordinary skill in the art, and all teachings in the prior art must be 
considered to the extent that they are in analogous arts." 71 "[Rejections on obviousness cannot 
be sustained by mere conclusory statements; instead, there must be some articulated reasoning 
with some rational underpinning to support the legal conclusion of obviousness." 72 In this case, 



67 Final Office action, dated May 30, 2008, page 4. 

68 Final Office action, dated May 30, 2008, page 4. 

69 Okamura, col. 2, lines 18-20, 42-46. 

70 Okamura, col. 2, lines 20-21; col. 6, lines 24-49; FIG.3. 

71 MPEP § 2143.01 

72 MPEP § 2143.01 quoting KSR International Co. v. Teleflex, Inc., 82 USPQ2d 1385, 1396 (2007); In re Kahn, 441 
F.3d 977, 988, 78 USPQ2d 1329, 1336 (Fed. Cir. 2006). 
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Appellants submit that the rejection under 35 U.S.C. § 103(a) is improper because the Office has 
not articulated any reasoning to support the legal conclusion of obviousness. 

As noted in Appellants Pre-appeal Brief, Appellants' representative repeatedly tried to 
contact the Examiner to request clarification of the rejection based on Okamura and to discuss 
the need for a new Office action. The Examiner did not return phone calls from Appellants' 
representative Frank Agovino and, when Mr. Agovino reached him, the Examiner refused a 
telephone conference. Thus, we have tried to work with the Examiner to clarify the faulty 
rejection, but he refused. Appellants are at a loss as to understanding the basis for combining the 
Sandvoss and Okamura patents to reject the claims. 

In particular, the final Office action admits that "Sandvoss does not explicitly indicate 
providing a participant state table indicating an activity state variable for each participant and 
assigning a predetermined weight to at least a participant for duration previously set." 73 
Although the final Office action states that Okamura teaches the elements 74 the final action fails 
to explain how Sandvoss and Okamura support an obviousness rejection. For example, there is 
no mention in the final action as to: (1) how the table of Okamura is related to Sandvoss or (2) 
how Okamura could be combined with Sandvoss or (3) how Okamura teaches modifying 
Sandvoss. As a minimum, the final rejection fails to provide some articulated reasoning with 
some rational underpinning to support the legal conclusion of obviousness. 75 Consequently, the 
rejection must be reversed by the Board. 

Specifically, the Examiner asserts "Okamura teaches providing a participant state table 
indicating an activity state variable for each participant and assigning a predetermined weight to 
at least a participant for duration previously set." 76 The Examiner jumps to the conclusion that it 
would have been obvious to one of ordinary skill in the art at the time of the invention to 
incorporate the teachings of Okamura into those of Sandvoss to make the system better 
organized and that the system will execute more efficiently if all the data were to be gathered 
into one data structure rather than multiple ones. 77 However, Applicants submit that Sandvoss 
and Okamura cannot be combined because they are non-analogous art which do not share a 
common structure or function in that Sandvoss relates to dynamic prioritizing of audio streams 

73 Final Office action, dated May 30, 2008, page 4. 

74 Final Office action, dated May 30, 2008, page 4. 

75 MPEP 2143, citing In re Kahn, 441 F.3d 977, 988, 78 USPQ2d 1329, 1336 (Fed. Cir. 2006). 

76 Final Office action, dated May 30, 2008, page 4. 

77 Final Office action, dated May 30, 2008, page 4. 
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whereas Okamura relates to selecting users to receive information distributed from a distribution 
database based on user attributes and feedback provided by the user. The Examiner's conclusion 
fails to articulate any reasoning with rational underpinning on how the cited references of 
Sandvoss and Okamura are analogous and could be combined. 

In addition, the Examiner has failed to articulate how the activity state (user feedback) 
taught by Okamura may be combined with Sandvoss. In this case, the differences in structure 
and function of the cited references are great and significant and cannot be combined as they are 
non-analogous art. 78 Sandvoss discloses teleconferencing where the multimedia streams with the 
highest priority level streams are actively transmitted and the weight used to determine priority is 
calculated from substream signals that are input to a process. 79 In contrast, the activity state 
(user feedback) of Okamura is analyzed by an activity state registering unit to either (1) register 
the name of user with the information activity management unit or (2) register a distribution 
unnecessary condition based on the distribution reason recorded inside the notified information 
ID with the distribution unnecessary condition storage unit. 80 As such, the references are non- 
analogous prior art because they do not share a common structure or function. Thus, the 
weighted, active transmission states of Sandvoss and the activity state (user feedback) of 
Okamura are different and cannot be combined without more explanation for the motivation or 
the manner of their combination. 

Moreover, as discussed above, there is no discussion in the Office action regarding how 
the activity state (user feedback) of Okamura can teach or suggest the claimed state table , or how 
the activity state (user feedback) of Okamura teaches or suggests the claimed activity state 
variable, much less the claimed activity state variable for each participant , and much less the 
claimed assignment of a predetermined weight to at least a participant for a duration previously 
set . Instead, the cited portions of Okamura teach an activity state registering unit for registering 
users in an item of user in the information activity management unit when notified by the users 
that the information was useful. 81 A close reading of Okamura reveals that a name or 
distribution condition registration occurs. When a user notifies the system that the information 

78 MPEP 2141.01(a) citing, In re Ellis, 476 F.2d 1370, 1372, 177 USPQ 526, 527 (CCPA 1973) (To rely on a 
reference under 35 U.S.C. 103, it must be analogous prior art. And, the Court has found "the similarities and 
differences in structure and function of the inventions [to] carry far greater weight" in determining what is analogous 
prior art.) 

79 Amendment I, filed September 10, 2008, page 11; Sandvoss, column 3, lines 53-58. 

80 Amendment I, filed September 10, 2008, page 11; Okamura, column 10, lines 31-47. 

81 Amendment I, filed September 10, 2008, page 11; Okamura, column 2, lines 37-41. 
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could be effectively utilized, the activity state registering unit registers a name of the user in the 
item of "user" within the information activity management unit specified by the notified 
information ID. 82 On the other hand, when the user notifies the system that the information is 
not required, the activity state registering unit registers the distribution unnecessary condition 
with the distribution unnecessary condition storage unit based on the distribution reason. 83 There 
is no discussion in these referenced portions of Okamura or elsewhere related to the recited 
invention, e.g., providing a participant state table indicating an activity state variable (including 
values and statistics associated with the participant's video signal and audio signal) for each 
participant and assigning a predetermined weight to at least a participant for a duration 
previously set, as recited in claim 10. 

As such, the Examiner has committed a clear error 84 by failing to provide a rational 
underpinning to support how the activity state (user feedback) of Okamura may be combined 
with Sandvoss to make obvious the activity state variable recited in claim 10. Therefore, the 
rejection of claim 10 under 35 U.S.C. § 103(a) must be withdrawn. 

(Hi). Sandvoss and Okamura can not be combined to produce the claimed invention 

The teachings of the references are not sufficient to render the claims prima facie obvious 
"if the proposed modification or combination of the prior art would change the principle of 
operation of the prior art invention being modified." 85 Moreover, even if the references could be 
combined, "[t]he mere fact that references can be combined or modified does not render the 
resultant combination obvious unless the results would have been predictable to one of ordinary 
skill in the art." 86 

In this case, there is no logical way that the teachings of Sandvoss could be modified by 
the teachings of Okamura in order to produce the claimed invention. For example, there is no 
logical way to combine the weight of Sandvoss with the activity state (user feedback) of 



82 Amendment I, filed September 1 0, 2008, page 1 1 ; Okamura, column 1 0, lines 3 1 -47. 
83 Amendment I, filed September 10, 2008, page 11; Okamura, column 10, lines 31-47. 

84 MPEP 2143, citing In re Kahn, 441 F.3d 977, 988, 78 USPQ2d 1329, 1336 (Fed. Cir. 2006). 

85 MPEP 2143.01, In reRatti, 270 F.2d 810, 123 USPQ 349 (CCPA 1959) 

86 MPEP 2143.01, citing KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385, 1396 (2007)("If a person of 
ordinary skill can implement a predictable variation, § 103 likely bars its patentability. For the same reason, if a 
technique has been used to improve one device, and a person of ordinary skill in the art would recognize that it 
would improve similar devices in the same way, using the technique is obvious unless its actual application is 
beyond his or her skill."). 
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Okamura, to provide a participant state table indicating an activity state variable for each 
participant and assigning a predetermined weight to at least a participant for a duration 
previously set, as recited in claim 10. As explained above, Sandvoss discloses teleconferencing 
where the multimedia streams with the highest priority level streams are actively transmitted and 
the weight used to determine priority is calculated from substream signals that are input to a 
process. And, Okamura teaches that information received from the system as to the usefulness 
of the information previously received is used to determined if the user will receive similar 
information in the future. The condition that Okamura would add to the system of Sandvoss is 
based on the user feedback which indicates that the user finds the information useful and would 
like to receive similar information (or that the user does not find the information useful and 
would not like to receive similar information). As such, the activity state (user feedback) of 
Okamura cannot be added to the system taught by Sandvoss to provide a participant state table 
indicating an activity state variable (including values and statistics associated with the 
participant's video signal and audio signal) for each participant and assigning a predetermined 
weight to at least a participant for a duration previously set. 

Furthermore, assuming without agreeing that Sandvoss and Okamura can be combined, 
their combination does not teach or suggest the features of the claimed invention nor would such 
results be predictable to one skilled in the art. At most, the combination of Okamura and 
Sandvoss would provide an activity state indicating user feedback. This user feedback to the 
teleconferencing of Sandvoss would merely indicate if the transmission of media were useful to 
the receiving user. This does not teach or suggest the features of claim 10. In addition, as 
discussed above in section A, Okamura does not cure the defects of Sandvoss since it does not 
teach a state table, much less a state table indicating an activity state variable for each 
participant, much less assigning a predetermined weight to at least a participant for a duration 
previously set . Thus, the Examiner has committed clear error and failed to make a prima facie 
case of obviousness. Therefore, the rejection of claim 10 under 35 U.S.C. § 103(a) must be 
withdrawn. 
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B(2). Claims 11-12 

Claims 11-12 depend from claim 10, and for at least the reasons above, Appellants 
respectfully submit that claims 1 1-12 are also patentable over Sandvoss in view of Okamura. 
Hence, the rejection of claims 1 1-12 under 35 U.S. C. § 103(a) should be withdrawn. 

B(3). Claim 13 

Claim 13 depends from claim 10 and adds further limitations to the updating of the 
activity state variable stored in the memory and the computing of the weight for each participant. 
In particular, claim 13 specifies that computing the weight for each participant includes 
"determining if the shown length of time is less than a first participant selection control 
parameter of a minimum shown time, and setting the weight to a second participant selection 
control parameter of a maximum weight if the shown length of time is less than the first 
participant selection control parameter." In an example explained in the present application, "[i]f 
the participant is being shown (step 184), it is determined whether the 

SecsSinceLastStartedShowing value is less than the Minimum Shown Time (step 192). If so, the 
weight is set to be MAXWEIGHT (step 194), which is a very large value, to ensure that this 
participant will be selected." 87 Thus, the computation required by claim 13 "guarantees that a 
participant, once selected for viewing, will be shown for at least the Minimum Shown Time (e.g., 
8 seconds)." 88 

As discussed above in connection with independent claim 10, Sandvoss and Okamura, 
alone and in combination fail to teach or suggest the participant selection control parameter of 
the present invention. Moreover, Sandvoss merely teaches that "the priorities of the streams are 
determined by a weighting function . . . comprising the amplitude loud(si,t) of an audio 
substream and a parameter A(si) which describes the last time when a stream has been active." 89 
Thus, Sandvoss fails to mention a parameter that determines whether a selected participant video 
stream has been shown for a minimum amount of time, as recited by claim 13. Okamura fails to 
cure the deficiencies of Sandvoss, since, as discussed above nothing in Okamura relates to the 
selection or transmission of a video stream. 



Application, page 20, lines 19-23. 
Application, page 20, lines 24-26. 
Sandvoss, col. 5, lines 36-45. 
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In view of these reasons and the reasons discussed above in connection with independent 
claim 10 from which claim 13 depends, claim 13 is patentable over Sandvoss in view of 
Okamura. Therefore, the rejection of claim 10 under 35 U.S.C. § 103(a) must be withdrawn. 

B(4). Claims 14-16 

Claims 14-16 depend from claim 13, and for at least the reasons above, Appellants 
respectfully submit that claims 14-16 are also patentable over Sandvoss in view of Okamura. 
Hence, the rejection of claims 14-16 under 35 U.S.C. § 103(a) should be withdrawn. 

B(5). Claim 18 

Claim 18 depends from claim 10, and for at least the reasons above, Appellants 
respectfully submit that claim 18 is also patentable over Sandvoss in view of Okamura. Hence, 
the rejection of claim 18 under 35 U.S.C. §103(a) should be withdrawn. 

(B)(6). Claim 24 

(i). Sandvoss and Okamura, alone or in combination, fail to teach each and every 
element of claim 24 

Claim 24 relates to "a method ... for intelligently selecting a single video stream from 
video streams originating from multiple participants of a multimedia network conference and 
sending the selected video stream to a client for viewing." 90 As recited by claim 24, the method 
comprises, among other operations, the combination of (a) "receiving a participant selection 
control parameter . . . having a static display constraint of a selection of a video signal," (b) 
"providing a participant state table . . . indicating an activity state variable for each participant" 
wherein the "activity state variable include[s] values and statistics associated with the 
participant's multimedia conference data"; and (c) "assigning a predetermined weight to at least 
one of the multiple participants for a duration specified by the static display constraint." 

As recited by claim 24, the method includes "receiving simultaneously multimedia 
conferencing data . . . including the plurality of video signals from the participants," and 

90 Application, page 3, lines 11-15. 
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"updating at least one of the activity state variables in the participant state table according to the 
participant events." The method includes "periodically computing a weight for each of the 
participants based on the activity state variable of said each participant and the participant 
selection control parameter" and, specifically, "assigning a predetermined weight to at least one 
of the multiple participants for a duration specified by the static display constraint." 
Accordingly, method includes "identifying a participant having a highest weight among the 
participants" and "selecting ... the one video signal corresponding to the identified participant 
having the highest weight for viewing by the client." 

For claimed subject matter to be prima facie obvious under 35 U.S.C. § 103(a) in view of 
prior art, the prior art references must individually or in combination disclose or suggest each of 
the requirements of the claim. 91 As similarly explained above in connection with claim 10, 
neither Sandvoss nor Okamura, alone or in combination disclose or suggest each of the 
requirements of claim 24. Thus, the rejection of claim 24 under 35 U.S.C. § 103(a) must be 
withdrawn. 

(i)(a). Sandvoss and Okamura, alone or in combination, fail to teach the participant 
selection control parameter having a static display constraint of claim 24 

As repeatedly pointed out by Appellants in Amendments G, H, and I, Sandvoss and 
Okamura fail to teach or disclose a "participant selection control parameter having a static 
display constraint on a selection of a video signal" as recited by claim 24. The method of claim 
24 specifically recites "when the conference is being set up, receiving a participant selection 
control parameter." The participant selection control parameter remains at the fixed value unless 
changed by the user. For example, the exemplary participant selection control parameters in 
Table 1 of the present application include a "Minimum Shown Time" and a "Minimum Shown 
Time If Active." The "Minimum Shown Time" control parameter specifies the minimum static 
period of time (8 seconds) that a selected participant's video stream will be displayed on the 
client stream. 92 The "Minimum Shown Time If Active" control parameter specifies the 
minimum static period of time (15 seconds) that a selected participant's video stream will be 



91 MPEP §§ 2143-2143.01. 

92 Application, page 18, Table 1, Row 2. 
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displayed on the client screen if the participant is still talking for this period of time." 93 Thus, the 
participant selection control parameter recited by claim 24 has a static value, as opposed to a 
dynamic value, and thus does not change unless specifically re-configured by a user. 

The Office incorrectly asserts that Sandvoss discloses the static display constraint 
requirement of claim 24 by teaching that the amplitude of an audio substream is used in a 
weighting function computed to select a multimedia signal. 94 However, as pointed out in 
connection with claim 10, the amplitude of the audio stream is a dynamic value, rather than a 
static value. Thus, Sandvoss fails to teach a "participant selection control parameter having a 
static display constraint on a selection of a video signal" as recited by claim 24. In fact, 
Sandvoss teaches away from using static display constraints received when the conference is set 
up to select a video stream by teaching that the priority of the streams are determined by a 
weighting function that is only dependent on dynamic values determined during the 
conference. 95 

Okamura does not cure the insufficient and contrary teachings of Sandvoss with regard to 
participant selection control parameter. As discussed above in conjunction with claim 10, 
Okamura teaches selecting users to receive information distributed from a distribution database 
based on user attributes and feedback provided by the user. 96 Okamura does not select 
information from one participant for transmission to a user. Moreover, nothing in Okamura 
teaches or suggests that the information includes a multimedia stream, such as a video stream or 
an audio stream. Accordingly, like Sandvoss, nothing in Okamura teaches or suggests a 
"participant selection control parameter having a static display constraint on a selection of a 
video signal" as recited by claim 24. 

(i)(b). Sandvoss and Okamura, alone or in combination, fail to teach the participant 
state table of claim 24 

The Office correctly acknowledges that Sandvoss fails to disclose each of the 
requirements of claim 24. Specifically, the final Office action admits that "Sandvoss does not 



93 Application, page 18, Table 1, Row 3. 

94 Final Office action, page 3. 

95 Sandvoss, col. 5, lines 44-46; See also, Section B(l)(i)(a). 

96 Okamura, col. 6, lines 11-25; FIG. 1; col, 7, lines 5-10; col. 12, lines 22-43. 
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explicitly indicate providing a participant state table indicating an activity state variable for each 
participant." 97 

As discussed above in Section B(l)(i)(b), although Okamura includes the term "activity 
state" as pointed out by the Office, 98 Okamura teaches a completely different type of "activity 
state" based on user feedback in contrast to the activity state variable recited in claim 24. 
Particularly, the activity state of Okamura refers to user feedback regarding information 
previously received by the user, which is completely different from the activity state variable of 
claim 24 which includes values and statistics describing a media signal transmitted by the user. 
Additionally, the activity state (user feedback) of Okamura is used to determine whether or not to 
send the user similar information items in the future, 99 rather than to determine whether 
information provided by a user is selected for transmitting to a client (i.e., claim 24 recites 
"selecting . . . video signal corresponding to the identified participant having the highest weight 
for viewing by the client). 

Accordingly, Okamura clearly fails to disclose or teach the activity state variable of claim 
24. Furthermore, even if Okamura was interpreted to teach such an activity state variable, 
Okamura fails to teach "a participant state table" indicating an activity state variable for each 
participant as required by claim 24. Okamura is entirely silent on a participant state table. After 
admitting that Sandvoss fails to disclose the participant state table, the Office has failed to point 
to even a suggestion thereof by Okamura. Thus, Okamura, like Sandvoss fails to disclose or 
suggest the "participant state table . . . indicating an activity state variable for each participant" 
recited by claim 24. 

(i)(c). Sandvoss and Okamura, alone or in combination, fail to teach the bridge server 
of claim 24 

As noted above, the Office correctly acknowledges that Sandvoss fails to disclose each of 
the requirements of claim 24. Specifically, the final Office action admits that "Sandvoss does 



Final Office action, dated May 30, 2008, page 4. 
Final Office action, dated May 30, 2008, page 4. 
Okamura, col. 6, lines 11-25; FIG. 1; col, 7, lines 5-10. 
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not explicitly indicate ... assigning a predetermined weight to at least a participant for duration 
previously set." 100 

Contrary to the Office's assertion, Okamura does not cure this failure of Sandvoss as 
Okumura does not teach or suggest "assigning a predetermined weight to at least one of the 
multiple participants for a duration specified by the static display constraint" as required by 
claim 24. The Office asserts that this feature is disclosed in Okamura at col. 2, lines 14-41. 101 
However, this portion of Okamura merely explains that predetermined information attributes 
(e.g., keyword contained in the information, information provider, location of information) 102 are 
used to characterize information and the information is distributed to users based on user 
attributers (e.g., name, address, subscribed newspaper/magazine) 103 and the activity state (user 
feedback) of the user discussed above. Nothing in this cited portion, or any other portion of 
Okamura, even suggests that predetermined weights are assigned to a user, much less that the 
weights are assigned a particular duration. Further, as discussed in Section B(2)(i)(a) Okamura 
fails to even disclose a parameter having static display constraint. Thus, Okamura, like Sandvoss 
fails to teach or suggest "assigning a predetermined weight to at least one of the multiple 
participants for a duration specified by the static display constraint" as required by the method of 
claim 24. 

Thus, Sandvoss and Okamura, alone and in combination fail to teach or suggest each 
and every element of the invention recited by claim 24. Accordingly, Appellants respectfully 
submit that claim 24 is non-obvious and patentable over the cited art and that the rejection of 
claim 24 under 35 U.S. C. § 103(a) should be withdrawn. 

(ii). The Office fails to provide articulated reasoning to support obviousness 

Additionally, "[t]he test for obviousness is what the combined teachings of the references 
would have suggested to one of ordinary skill in the art, and all teachings in the prior art must be 
considered to the extent that they are in analogous arts." 104 "[Rejections on obviousness cannot 
be sustained by mere conclusory statements; instead, there must be some articulated reasoning 



Final Office action, dated May 30, 2008, page 4. 
Final Office action, dated May 30, 2008, page 4. 
Okamura, col. 2, lines 18-20, 42-46. 
Okamura, col. 2, lines 20-21; col. 6, lines 24-49; FIG.3. 
MPEP § 2143.01 
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with some rational underpinning to support the legal conclusion of obviousness." 105 In this case, 
Appellants submit that the rejection of claim 24 under 35 U.S. C. § 103(a) is improper because the 
Office has not articulated any reasoning to support the legal conclusion of obviousness as 
discussed above in Section B(l)(ii). 

(Hi). Sandvoss and Okamura can not be combined to produce the claimed invention 

The teachings of the references are not sufficient to render the claims prima facie obvious 
"if the proposed modification or combination of the prior art would change the principle of 
operation of the prior art invention being modified." 106 Moreover, even if the references could 
be combined, "[t]he mere fact that references can be combined or modified does not render the 
resultant combination obvious unless the results would have been predictable to one of ordinary 
skill in the art." 107 

As discussed in Section B(l)(iii) regarding claim 10, there is no logical way that the 
teachings of Sandvoss could be modified by the teachings of Okamura in order to produce the 
claimed invention. Furthermore, assuming without agreeing that Sandvoss and Okamura can be 
combined, their combination does not teach or suggest the features of the claimed invention nor 
would such results be predictable to one skilled in the art. At most, the combination of Okamura 
and Sandvoss would provide an activity state indicating user feedback (activity state) which is 
completely different from the claimed invention. Thus, the Examiner has committed clear error 
and failed to make a prima facie case of obviousness. Therefore, the rejection of claim 24 under 
35 U.S. C. §103 (a) must be withdrawn. 

B(7). Claims 25-29 

Claims 25-29 depend from claim 24, and for at least the reasons above, Appellants 
respectfully submit that claims 25-29 are also patentable over Sandvoss and Okamura. Hence, 
the rejection of claims 25-29 under 35 U.S.C. § 103(a) should be withdrawn. 

105 MPEP § 2143.01 quoting KSR International Co. v. Teleflex, Inc., 82 USPQ2d 1385, 1396 (2007); In re Kahn, 
441 F.3d 977, 988, 78 USPQ2d 1329, 1336 (Fed. Cir. 2006). 

106 MPEP 2143.01, In re Ratti, 270 F.2d 810, 123 USPQ 349 (CCPA 1959) 

107 MPEP 2143.01, citing KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385, 1396 (2007)("If a person of 
ordinary skill can implement a predictable variation, § 103 likely bars its patentability. For the same reason, if a 
technique has been used to improve one device, and a person of ordinary skill in the art would recognize that it 
would improve similar devices in the same way, using the technique is obvious unless its actual application is 
beyond his or her skill."). 
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B(8). Claim 30 

Claim 30 depends from claim 24 and adds further limitations to the updating of the 
activity state variable the computing of the weight for each participant as performed by the 
method therein. In particular, claim 30 specifies that computing the weight for each participant 
includes "if the value for the activity state variable is less than a first value for the participant 
selection control parameter for a minimum time that a selected participant's video signal will be 
displayed on the client screen, setting the weight to a second value of the participant selection 
control parameter for a maximum weight." In an example explained in the present application, 
"[i]f the participant is being shown (step 184), it is determined whether the 
SecsSinceLastStartedShowing value is less than the Minimum Shown Time (step 192). If so, the 
weight is set to be MAXWEIGHT (step 194), which is a very large value, to ensure that this 
participant will be selected." 108 Thus, the computation required by claim 13 "guarantees that a 
participant, once selected for viewing, will be shown for at least the Minimum Shown Time (e.g., 
8 seconds)." 109 

As discussed above in connection with independent claim 24, Sandvoss and Okamura, 
alone and in combination fail to teach or suggest the participant selection control parameter of 
the present invention. Moreover, Sandvoss merely teaches that "the priorities of the streams are 
determined by a weighting function . . . comprising the amplitude loud(si,t) of an audio 
substream and a parameter A(si) which describes the last time when a stream has been active." 110 
Thus, Sandvoss fails to mention a parameter that determines whether a selected participant video 
stream has been shown for a minimum amount of time, as recited by claim 30. Okamura fails to 
cure the deficiencies of Sandvoss, since, as discussed above nothing in Okamura relates to the 
selection or transmission of a video stream. 

In view of these reasons and the reasons discussed above in connection with independent 
claim 24 from which claim 30 depends, claim 30 is patentable over Sandvoss in view of 
Okamura. Therefore, the rejection of claim 30 under 35 U.S.C. § 103(a) must be withdrawn. 



Application, page 20, lines 19-23. 
Application, page 20, lines 24-26. 
Sandvoss, col. 5, lines 36-45. 
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B(9). Claims 31-35 

Claims 31-35 depend from claim 24, and for at least the reasons above, Appellants 
respectfully submit that claims 31-35 are also patentable over Sandvoss and Okamura. Hence, 
the rejection of claims 31-35 under 35 U.S.C. § 103(a) should be withdrawn. 
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VIII. CONCLUSION 

For the reasons stated above, Appellants respectfully request that the Office's rejections 
be reversed and that claims 10-16, 18, and 24-35 be allowed. 



Respectfully submitted, 

/Frank R. Agovino/ 

Frank R. Agovino, Reg. No. 27,416 
SENNIGER POWERS, LLP 
100 North Broadway, 17th Floor 
St. Louis, Missouri 63102 
(314) 345-7000 
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IX. CLAIMS APPENDIX 

Claims 1-9. (canceled) 

Claim 10. A system for conducting a multimedia conference, comprising: 

a plurality of participants each providing multimedia conferencing data including a video 
signal and an audio signal; 

a client in conference with the participants, the client capable of receiving the video 
signal corresponding to one of the participants at a time; 

a participant selection control parameter stored in a memory for tuning a video switching 
stream behavior, wherein the participant selection control parameter affects the outcome of a 
weight computation, said participant selection control parameter being received when the 
multimedia conference is set up, said participant control selection parameter having a static 
display constraint on a selection of a video signal; 

a participant state table stored in a memory and indicating an activity state variable for 
each participant, said activity state variable including values and statistics associated with the 
participant's video signal and audio signal; and 

a bridge server connected to the participants through a network and having a point-to- 
point connection with the client, the bridge server assigning a predetermined weight to at least 
one of the plurality of participants for a duration specified by the static display constraint, 
receiving simultaneously the multimedia conferencing data including the video signal from each 
of the participants, updating the activity state variable stored in the memory for each participant 
in the participant state table according to changes in the data information and the control 
information of the participant's video signal and audio signal, periodically computing a weight of 
said each participant based on the activity state variable of said each participant and the 
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participant selection control parameter, identifying a participant having a highest weight among 
the participants, and selecting from the received multimedia conferencing data the video signal 
corresponding to the identified participant having the highest weight for transmission to the 
client for viewing. 

Claim 1 1 . A system as in claim 10, wherein the plurality of participants and the bridge server are 
connected through a multicast network. 

Claim 12. A system as in claim 10, wherein the bridge server further transmits to the client an 
audio stream containing a mixture of the audio signals from the participants of the network 
conference. 

Claim 13. A system as in claim 10, wherein updating the activity state variable stored in the 

memory for each participant includes: 

determining a shown length of time for which the video signal of said each participant 

has been shown to the client if said each participant is currently being shown to the client; and 
wherein the computing of the weight of said each participant includes: 
determining if the shown length of time is less than a first participant selection control 

parameter of a minimum shown time, and setting the weight to a second participant selection 

control parameter of a maximum weight if the shown length of time is less than the first 

participant selection control parameter. 
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Claim 14. A system as in claim 13, wherein updating the activity state variable stored in the 
memory for each participant includes: 

determining a talking length of time for which said each participant has been talking; and 

wherein computing the weight of said each participant includes: 

if the shown length of time is more than the first participant selection control parameter, 
then determining if the talking length of time is less than a third participant selection control 
parameter of a minimum shown time if active, and setting the weight to the second participant 
selection control parameter if the talking length of time is less than the third participant selection 
control parameter. 

Claim 15. A system as in claim 14, wherein the computing of the weight of said each participant 
by the bridge server includes: 

if the shown length of time is more than the first participant selection control parameter 
and the talking length of time is more than the third participant selection control parameter, then 
determining if the shown length of time is less than a fourth participant selection control 
parameter of an active cycle time indicating an upper limit of time for showing a video signal of 
an active talking participant, and if the shown length of time is less than the fourth participant 
selection control parameter, then setting the weight to a value increasing the likelihood that said 
each participant will be identified as having the highest weight. 

Claim 16. A system as in claim 15, wherein the computing of the weight of said each participant 
includes: 
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if the shown length of time is more than the first participant selection control parameter 
and the talking length of time is more than the third participant selection control parameter and 
the shown length of time is more than the fourth participant selection control parameter, then 
setting the weight to a value decreasing the likelihood that said each participant will be identified 
as having the highest weight. 

Claim 17. (canceled) 

Claim 18. A system as in claim 10, wherein the multimedia conferencing data received by the 
bridge server include a combined video stream having substreams corresponding to the 
participants, and wherein the bridge server demultiplexes the combined video stream into a 
plurality of individual video signals each including one of the substreams in the combined video 
stream. 

Claims 19-23. (canceled) 

Claim 24. A method for selecting one video signal from a plurality of video signals for 
forwarding to a client, each video signal corresponding to a participant of multiple participants of 
a multimedia conference, said method comprising: 

when the conference is being set up, receiving a participant selection control parameter for 
the multimedia conference, said participant selection control parameter having a static display 
constraint of selecting the one video signal; 

assigning a predetermined weight to at least one of the multiple participants for a duration 
specified by the static display constraint; 



31 



receiving simultaneously multimedia conferencing data from the multiple participants, 
the multimedia conference data including the plurality of video signals from the participants; 

monitoring participant events of the multimedia conference, said participant events 
associated with the multimedia conferencing data of the participants, said participant events 
being generated in response to changes in the data information and the control information of the 
multimedia conferencing data; 

providing a participant state table associated with the multimedia conference indicating 
an activity state variable for each participant of the multimedia conference, said activity state 
variable including values and statistics associated with the participant's multimedia conference 
data; 

updating at least one of the activity state variables in the participant state table according 
to the participant events; 

periodically computing a weight for each of the participants based on the activity state 
variable of said each participant and the participant selection control parameter; 

identifying a participant having a highest weight among the participants; and 

selecting from the received multimedia conferencing data the one video signal 
corresponding to the identified participant having the highest weight for viewing by the client. 

Claim 25. The method of claim 24, wherein the multiple participants are connected to a bridge 
server through a multicast network. 

Claim 26. The method of claim 25, further comprising transmitting to the client an audio stream 
containing a mixture of audio signals from the multiple participants of the network conference. 
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Claim 27. The method of claim 24, wherein the computing the weight comprises: 

determining whether each participant's activity state variable indicates the participant is 
currently being shown to the client, wherein if a first participant's activity state variable indicates 
the first participant is not being shown to the client and the first participant is talking, a value for 
the participant selection control parameter for a maximum time that can elapse before a second 
participant who is not talking is selected is added to the first participant's weight. 

Claim 28. The method of claim 24, wherein the updating the activity state variable comprises: 
determining a value for the activity state variable for a length of time for which said each 

participant has been shown to the client if said each participant is currently being shown; and 
determining whether said each participant is talking; 
wherein the computing the weight comprises: 

if the participant is not talking and the value of the activity state variable is less than a 
first value for the participant selection control parameter for a time period that each participant's 
video signal is displayed if none of the participants is talking, setting the weight to a second 
value for the participant selection control parameter for enhancing a likelihood that the 
participant will be selected; and 

if the participant is not talking and the value of the activity state variable is greater than 
the first value for the participant selection control parameter, setting the weight to a third value 
for the participant selection control parameter for enhancing a likelihood that the participant will 
not be selected. 
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Claim 29. The method of claim 24, wherein the updating the activity state variable comprises: 

determining a value for the activity state variable indicating whether said each participant 

is not sending video; and 

wherein computing the weight for each of the participants comprises: 

if the value for the activity state variable indicates said each participant is not sending 

video, setting the weight to a value of the participant selection control parameter for a minimum 

weight. 

Claim 30. The method of claim 24, wherein the updating the activity state variable comprises: 
determining a value for the activity state variable for a length of time for which said each 

participant has been shown to the client; and 

wherein computing the weight for each of the participants comprises: 

if the value for the activity state variable is less than a first value for the participant 

selection control parameter for a minimum time that a selected participant's video signal will be 

displayed on the client screen, setting the weight to a second value of the participant selection 

control parameter for a maximum weight. 

Claim 3 1 . The method of claim 24, wherein the multimedia conference data includes a 
combined video stream containing multiple substreams, each substream corresponding to a video 
signal of one of the multiple participants, and wherein the receiving includes demultiplexing the 
combined video stream into the plurality of video signals, each video signal including one of the 
substreams of the combined video stream. 
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Claim 32. The method of claim 24, wherein the participant selection control parameter includes 
at least one member of a group comprising: a maximum weight; a minimum weight; a minimum 
time that a selected participant's video signal will be displayed on a client screen; a minimum 
time that a selected participant's video signal will be displayed on the client screen if the 
participant is still talking; a time period between computing weights and switching video; a time 
period that an actively talking participant's video signal is displayed on the client screen if only 
the actively talking participant is talking; a time period that each participant's video signal is 
displayed if none of the participants is talking; a value used to enforce a time period that the 
actively talking participant's video signal is displayed on the client screen if only the actively 
talking participant is talking, said value being less than a maximum time that can elapse before a 
participant who is not talking is selected; a value used to enforce a time period that each 
participant's video signal is displayed on the client screen if none of the participants is talking, 
said value being less than the time period that each participant's video signal is displayed if none 
of the participants is talking; and a maximum time that can elapse before a participant who is not 
talking is selected. 

Claim 33. The method of claim 24, wherein the activity state variables includes at least one of a 
group comprising: a time value for which the participant's video signal was last displayed on a 
client screen; a time value for which the participant's video signal has been displayed on the 
client screen; a time value since the participant started talking; a time value since the participant 
stopped talking; a Boolean value indicating whether the participant is currently talking; a 
Boolean value indicating whether the participant is currently sending a video signal; and a 
Boolean value indicating whether the participant's video signal is currently being displayed on 
the client screen. 
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Claim 34. The system of claim 10, wherein the participant selection control parameter includes 
at least one member of a group comprising: a maximum weight; a minimum weight; a minimum 
time that a selected participant's video signal will be displayed on a client screen; a minimum 
time that a selected participant's video signal will be displayed on the client screen if the 
participant is still talking; a time period between computing weights and switching video; a time 
period that an actively talking participant's video signal is displayed on the client screen if only 
the actively talking participant is talking; a time period that each participant's video signal is 
displayed if none of the participants is talking; a value used to enforce a time period that the 
actively talking participant's video signal is displayed on the client screen if only the actively 
talking participant is talking, said value being less than a maximum time that can elapse before a 
participant who is not talking is selected; a value used to enforce a time period that each 
participant's video signal is displayed on the client screen if none of the participants is talking, 
said value being less than the time period that each participant's video signal is displayed if none 
of the participants is talking; and a maximum time that can elapse before a participant who is not 
talking is selected. 

Claim 35. The system of claim 10, wherein the activity state variables includes at least one of a 
group comprising: a time value for which the participant's video signal was last displayed on a 
client screen; a time value for which the participant's video signal has been displayed on the 
client screen; a time value since the participant started talking; a time value since the participant 
stopped talking; a Boolean value indicating whether the participant is currently talking; a 
Boolean value indicating whether the participant is currently sending a video signal; and a 
Boolean value indicating whether the participant's video signal is currently being displayed on 
the client screen. 
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EVIDENCE APPENDIX 

None. 



RELATED PROCEEDINGS APPENDIX 

None. 
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